y-T' ■ 



in is* Is? 

■ \ 

A8TH3B 
TITLE 

P8B DUE 
NOTE 



SBESL PRICE 
DBSCBIPIOHS 



BBSOI3 



SS 03 0 525 



kill as haw, P. Gerald; Okey, Jases B. 
A rest of vhe Integrated Science Process Skills for 
Secondary Science Students, 
ape 30 

Iftjs; Paper presented at the annual aeeting of the 
National association for Research in Science Teaching 
(53ri, Boston, HA , April 11-t 3, 1980). 

MP01/PC 01 Pins Postage. 

Academic Achievement: *Educat ional Sesearch; Process 
Education: ^Science Tests: Secondary Education; . 
« ^Secondary School Science; *Skill Develop sent; 
Student Evaluation: *Test construction 

ABSZRACI 

Presented is a study in which the authors developed in- 
valid and reliable test of 1 integrated science process skills for 
students in secondary schools. The field trials are described and tha 
reliability is given as .89 across ability levels, socioeconomic 
^.levels, gender, and race. Secoaaended uses of the test for teachers, 
researchers, and evaluators are included. (S&) 
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A TEST OF THE INTEGRATED SCIENCE PROCESS SKILLS 
FOR SECONDAR* SCIENCE STUDENTS 

Many science curriculum guides and textbooks have as important outcomes 
the acquisition of the integrated science process skills. Students using 
these materials are expected to develop such skills as stating hypotheses, 
operationally defining variables, designing investigations, and interpreting . 
data in addition to mastering the content of the courses. But if these inte- 
grated skills are emphasized by instructors, then they need a means to assess 
the acquisition of the skills. The purpose of this project was to develop a 
valid and reliable test of the integrated science process skills that could be 
used by teachers, e valuators , and researchers with secondary science students. 

Background Literature 
During the 1960's and 1970 's the move to incorporate more inquiry and 
investigations in science classes was accompanied by efforts to measure these 
outcomes. Walbesser (1965) and others associated with the Science-A Process 
Approach curriculum developed a test of the basic and integrated process skills 

f 

especially intended for elementary children using the curriculum. McLeod, 
Berkheiroer, Fyffe, and Rob i son (1975) developed the Group Test of Four Processes 
to measure the skills of controlling variables, interpreting data, formulating 
hypotheses, and operationally defining. This test was designed to measure 

< 

these skills as defined by the Science-A Process Approach elementary school 
science curriculum. Ludeman (1975), building on the work of McLeod, et al. 
developed The Science Processes Test . Again, this test was aimed at the 
elementary grade levels. Riley (1972) developed the Test of Science Inquiry 
Skills (TSIS) for fifth grade students. The TSIS measures the skills of 
identifying and controlling variables, interpreting data, predicting, and 
inferring as defined by the Science Curriculum Improvement Study elementary 
science program. 
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In an attempt to separate * process skills test from a specific curriculum, 
Molitor and George (1976) designed a test to assess the inquiry skills of 
inference and verification. Their test was designed for grades four, five, 
and six. Tannenbaum (1968) developed a test for use in grades, seven, eight, 
and nine. This instrument, Test of Science Processes , assessed the skills of 
.observing, comparing, classifying, quantifying, measuring, experimenting, 
inferring, and predicting. 

In addition to tests for students, tests of the process skills have been 
developed for teachers. Burns (1972) developed a test to measure the acquisi- 
tion of the integrated science process skills by undergraduate elementary 
education majors. A guide prepared to help teachers leam the process skills 
associated with Science- A Process Approach includes appraisal items for 

teachers (American Association for the Advancement of Science, 1967). 

\ 

Of the science curriculum projects for secondary schools, only the 
Biological Sciences Curriculum Study has a test specifically designed to 
measure process skill competence. This instrument, Processes of Science Test , 
measures how competent students are in the processes of science. Although the 
test purports to be general rather than biological science specific, the test 
items show an exclusive use of biological concepts and examples. 

The preceding references to test construction all come from the period 
of the sixties and seventies. But assessment of the process or problem-solving 
component of science learning can be traced back much further. Champagne and 
Klopfer (1977) review reports and descriptions of process-oriented problem 
solving that begin as early as 1916. In these early studies, attempts to 
measure knowledge of problem solving or the methods of science appear to com- 
bine tests of specific skills (e.g., arranging data in sequence) and of 
scientific practices (e.g., suspending judgment). 
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The search of available process skills tests showed the need for a test 
geared to secondary students and not associated with any particular science 
curriculum. 



The science process skills selected for testing are those associated with 
planning, conducting, and interpreting results from investigations. Usually 



referred to as the integrated science processes, they include formulating 
hypotheses 9 operationally defining, controlling and manipulating variables 9 
planning investigations, and interpreting data (Liverroore, 196*4). Objectives 
(see Figure 1) for each selected process skill were written. These objectives 
collectively lead to the ultimate task or planning and conducting an investi- 
gation. 



A major consideration in developing a propess skills test is that of test 



format. While one wants students to demonstrate the integrated skills, the 
problem of using hands-on procedures to assess skill acquisition by groups of 
students can be a burdensome task. Faced with five or more classes and thirty 
or so students per class, science teachers need a means of measuring process 
skill competency that can be administered efficiently and objectively. There- 
fore, a decision was made to utilize a paper and pencil group testing format. 

Three multiple-choice test items (each with four alternatives) were written 
for each of the nine outcomes shown in Figure 1. Objective A had four parts 
so a total of 36 items were written. Content was chosen from a variety of 
science fields so that the items were not based on a detailed knowledge of any 
subject field. 

The Test of Integrated Process Skills (TIPS) that resulted from the 
process described above was su* mitted to a panel of four science educators to 
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Procedures 



Insert Figure 1 About Here * 
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establish the content validity of the test, the object ivity of the scoring key, 
and the clarity of the items. To determine content validity, the panel was 
given a copy of the test items and a list of the outcomes and asked to identify 
which test items corresponded to which outcomes. The panel also completed the 
test so the scoring key could be verified- Comments on item clarity along with 
the validity and objectivity data were used to make modifications in some items. 
For a total of responses (4 raters X 36 items), the experts agreed with 
the test developers on the assignment of test items to objectives 95% of the 
time and agreed with the scoring of the test 97% of the time. This concurrence 
of raters was taken as evidence of content validity and. objectivity of scoring. 

■ Field Trial 1 

The first field trial was conducted in a community of approximately 50,000. 
The racial composition of the schools is about 35% minority and 65% white. 
Busing brings in rural students, thus resulting in school populations of mixed 
rural and urban students. Although random selection of subjects was not 
possible, classes were selected that reflected all facets of the school popu- 
lation. Approximately 100 students each from grades seven, nine, and eleven 
in two schools were tested in the first trial. The purposes of the first trial 
were to establish time to complete the test, to determine clarity of test 
directions, and to establish difficulty and discrimination indices for the 
test items. 

* The test was administered to a sample of 308 students.- The mean score 
was 18.75 and the standard deviation was 7.59. Reliability (Cronbach's a) 
based on the total sample was equal to .88. The mean item discrimination 
index was .39 and the average item difficulty index was 52%. 



Test Revisions t 

An assessment of the test readability level (Heilman, 1972) showed it to 
be higher than desired (about 11th grade). Words such as experiment , variable , 
investigation, and manipulated are used frequently in the test and caused the 
readability to be high. Further, the correlation (point bi serial) of item 
scores with total test scjbres showed that several items were not discriminating 
well between high and low scoring students. These two factors (readability 
level and item discrimination) led to revisions in about fcqjf of the test items. 
Sentences were shortened and items that seemed hard to read were clarified. 
Figure 1 provides examples of the test items included in the final version of 
the test. 

Field Trial 2 

The revised version of the test was administered in a large school system 
in a community of approximately 100,000 in an adjoining state. A sample of 
roore than 700 students from grades seven, nine, and eleven was selected. The 
racial and rural/urban composition of this system is similar to that of the 
system used for the first field trial. Again classes were selected to provide 
representation across ability levels, socioeconomic levels, gender, and race. 

Results 

The revised version of the test was administered to a sample of 709 
students. The mean of the sepond version of the test is 18.99 and the standard 
deviation is equal to 7.60. With no specific training in the integrated process 
skills, students scored on the average about mid-range on the 36- item test. 
This allows considerable room to record increases in achievement for students 
receiving instruction with an emphasis on process skills. 
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Reliability of tha test (Cronbach*s a) is equal to .89. The mean item 
discrimination index is .10 and the average item difficulty index is 53%. 
Each of these three test characteristics is within acceptable limits for 
reliable tests (Payne, 1974). An estimate of the readability index is 9.2. 
Although this value of readability level seems a bit high, it results from 
the necessary use of multiple syllable words associated with investigating. 
Table 1 gives descriptive data by 'grade level and overall for the final version 
of the test. Based on these results from the second field trial, no further 
changes in the test were considered necessary. 

\ 
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Insert Table 1 About Here 

Figure 2 summarises descriptive characteristics of the Test of Integrated 
Process Skills . 

Insert Figure 2 About Here 
Conclusions 

It should be emphasized that the purpose of this project was not to make 
inferences about the level of process skill attainment of a specific group of 
students. The tryout students had not had specific or intensive instruction 
in process skills. The scores reflect process skill capabilities of students 
that result from a range of content-oriented science teaching practices. 
Instead f the intent of the project was to develop a measure of integrated 
process skill achievement referenced to a specific set of objectives. The 
Test of Integrated Process Skills (TIPS) appears to be a valid and reliable 
measure of process skill achievement for students in the seventh to twelfth 
grade range. 



8 



a. 



Use of the teat* is recommended to teachers, researchers, end evaluators 

in ways such as the following: 

1. Research studies in which the dependent variable is student acqui- 
sition of the science process skills. 

2. Teaching skills research where the effectiveness of certain teaching 
practices is measured by process skill achievement. 

3. Assessing process skill competency by classroom teachers where the 
process skills are an important outcome of science instruction. 

H, Tests of the effectiveness of materials or modules designed to aid 
students in learning science process skills. 

5. An alternative to a laboratory procedure as a way to assess process 
skill acquisition. 
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^Copies of the test for use in research and evaluation studies can be obtained 
from the authors. 
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given « descriptian of ftS 
invest igst ion • identify the 
independent, dependent, *ad 
controlled variables and the 
hypothesis tela* tiit*4. 



Given a description of en 
investigation, identify how 
the vtrUMu ere operationally 
define*. 



Given * prahiea with a 
dependent variaMe specified. 
Identify variables vhich stay 
Affect it. 



D, 



Giv*a e profclea with dtpettdsat 
variables specified and a lilt 
of possible independent vari- 
ables, identify a testable 
hypothesis* 



Given a verbally described 
variable, select a suitable 
operational dsfinitioo for it. 



Given a problem with e dependant 
vsriablf specified, identify a 
testable hypothesis. 



Saaple ?tet tteje 

Sarah wanted to f lad out if te*pe*et\we baa Oft effect on tbo gwth of fcre»4 
■old. She are* the moid In nine oootainere containing the eaae aejount and 
type of nutrients. Three oortainera were kept at 0*C, three «are kept at 
tc*c, and three ware kept at raft teanereture (about 2TC). The coatainere 
we enaftinod and the growth of the bread meld nee recorded at the, end of 
four days, the dependent e* responding variable Us 

X. ' growth of bread aoid 

2. eaouot of nutrients in each container 

3. teaepereture of the containers 

H. sunber of containers ftt each teeeperature 

The superintendent i» concerned about the eccidtuta in ecteela. He aakee 
the hypothesis that safety advertising will re4i*a school accidents. He 
decides to teat the hypothesis is four middle schools, tech school 
use a different number of safety posters to sae if the ouefcor of accidents 
are reduced. Each school nuree wiU Weep a record of students that com 
to the office because of an accident. Sow is safety advertising ■eesured 
in this study? 

I. number of accidents reported to the nurse 

2. nuftber of aiddle schools involved 

3. nuieber of safety posters in each school 
h. nuaber of accidents in the school 

Sua wants to find out what night affect the length of beta seedlings* She 
places a bean wrapped in aolst tissue paper to each of ten identical teat 
tubas. She puts the other fiva tubes in a rack in a sunny window. She puts 
the other five tubes in a rack in ft funny window* She puts *he other five 
tubes on a rack in a dark refrigerator. She measures the lengths of the 
bean seedlings in esch group after one week, Which of the following variables 
might affect the length of the bean seedlings? 

1. tenpersature and moisture 

2. moisture and length of tast tubas 

3. light and temperature t 
*« light and amount of tiae 

A student has beeo playing with a water rocket . He can change the smount 
of veter in the rocket end the angle at which he reitsses the rockets He 
can also change the weight of the rocket by adding send in the nose cone. 
He wants to see what might affect the height to which the rocket will rise. 
Which of the following is a hypothesis he could test? 

1, Kocktt* with vara water will rise higher than rockets with cold water. 

2, Kockste with four tail fins will rime higher than rockets with twa tail 
fins, 

%; ftocktts with pointed nose cones will rise higher than rockets with 

rounded nose cones*. 
**, ftocktts with more water will rise higher than rockets with less water. 

The effect of erne re is e en pulse rate is studied by a science class. 
Students do different numbers ef Jumping Jacks and then manure pulse rate . 
Ont group does jumping jacks for one minute. * smeend group does them for 
two minutes, A third group jumps for three minutes, a fourth group dots 
not jump. How would you net sure the pulse rata in this study? 

1. by counting the nunber of jucping jacks for one minute 

2. by counting the nunber of heart bsats in one minute 

3. by counting the nuaber of jumping jacks done by each group 
**. by counting the number of exarciss* for each group 

Some chickens lay an egg almost evmry day. Other chicktft* produce few tgjes. 
A study is planned to exeoine factors that might affect the nushar of egg* 
produced by chicksos. Which of the following is HOT s suitable hypothesis 
for the stuuy? 



1. 

?♦ 
3. 



Hort eggs are produced by chickens that recsive mope hours of light , 
The more eggs produced by chickens the aore waight they lose. 
The larger the cage for chickans the acre eggs they will product. 
The more protein there is in the feed the aore tgfs produced. 



Figure 1. Objectives and sample test items for TIPS. 
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dv*& a description of as io- 
¥«ttii»tioo *»4 ofctoinAd d*t«» 
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tbo dst*. 



Jala wafers trim offoo*« tbo tio* it tikoo id* «i*** to «oit. «• tfcioko 
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tbo iM cute am «U fsctor* that ml&tet offtet tbo oolti&g *i»o. it 
tSftiUy (Socio** to toot tbo lypotteiit tbst tbA ***po of cs let cub* 
offset* tto tim it tokoo to **lt. '*i*k «**£*? abou*d*4ol» t«l*et to 
toot Mis hypcthosis? 

tta* fi** ico cub**, ooeh witb * difforoR ibtpo sod *sigfct. ** fit* 
idjotieol oootmiMPOt ill *t tbo sajss to f orots* ** Oboomo tb* 
aoltifif, of *b* ioo cvfc**. 

0o* fiv* ico eubss. *U b**iof tbs mo* obopo, bu* **cb tmriaf s dif- 
ferent wolfbt. tfs* fiwo idootioAi o o*t*l o *r o, *U *t tb* mm t**s?**ft 
tun, Moon* tb* Mltis« tioo of tbo ioo cube*. 
Ua* fiv* ioo cub** f aU hovlac tb« **m* w*i#t, but see* boviof * 
different *b*P*. Us* five identical oo*t*Ao*r* f oil at tbo on. 
t^twi. 3be*r«* tb* Mitisf tim of tbo ico cebee. 
*. Uoo five ico cube*, ftU bovici tbo ease «oi&*C, but **ab bavtag * 
different shape. Uo* firm idestieal eoAtoi&ws* osch «t o different 
tooeorowo. Obe*r*e tbo fleltiA* tiot of tbo ico cube*. 

A eci**c* cleo* uoo studying pressor* *ad «o&aee *itb baiiooo*. tboy did 
or eic*ori«*aat ia whicb thoy cheated tbo preaoaro c* o baUooo oad the* 
seesured its volute, tbo results or* fi«*a is the table. 
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Figure 1. Continued 
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Table 1 

Descriptive Statistics for the Test of Integrated Process Skills (TIPS) 



Grade 


N 


Mean 


S.D. 


Reliability 


7 


250 


15.39 


6.76 


.84 


3 


255 


19.83 


6.46 


.84 


11 


204 


22.29 


8.08 


.91 


OVERALL 


709 


18.99 


7.60 


.89 



Process skills tested: 



Number of items: 

Response format: 

Recommended grade levels: 

Time to complete test: 
(average) 



Test readability: 
Discrimination index: 



Difficulty index: 



identifying variabiles 
operationally defining 
-identifying testable hypotheses 
experimental design 
graphical analysis of data 

36 

multiple-choice; four alternatives 
7-12 



grade 7 — 
grade 9 — 
grade 11 — 

9.2 

Average 
.HO 



Average 
53% . 



50 minutes 
35 minutes 
25 minutes 



0 

.21 
.11 



0 
21 

Ml 
61 



.20 (2 items) 
.40 (12 items) 
.60 (22 items) 

Range 

20 (2 items) 
10 (6 items) 
50 (15 items) 
30 (13 items) 



Reliability: 



.89 ( Cronbach ' s a) 



Figure 2. Characteristics of the Test of Integrated Process Skills 
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